Windowing Models for Abstractive Summarization of Long Texts

Neural summarization models have a fixed-size input limitation: if text length surpasses the model’s maximal length, some document content (possibly summary-relevant) gets truncated. Independently summarizing windows of size disallows for information flow between and leads to incoherent summaries. We propose windowing neural abstractive (arbitrarily) long texts. extend sequence-to-sequence model augmented with pointer generator network by (1) allowing encoder slide over different (2) sharing decoder retaining its state across windows. explore two variants: Static Windowing precomputes number tokens generate from each window (based on training corpus statistics); in Dynamic learns emit token signaling shift next window. Empirical results render our effective intended use-case: texts relevant not bound beginning.